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RETENTION  OF  DIGITAL  SKILLS:  COMMAND  POST  OF  THE  FUTURE 

EXECUTIVE  SUMMARY 


Research  Requirement: 

As  the  U.S.  Anny  continues  to  provide  Soldiers  with  the  next  generations  of  digital 
systems,  the  digital  skills  needed  to  employ  these  systems  is  evolving  and  may  require  the 
acquisition  and  retention  of  new  skills.  This  is  especially  true  for  the  digital  systems  used  in 
tactical  operations  centers  (TOC)  and  command  posts.  A  Soldier’s  inability  to  utilize  previously- 
learned  digital  skills  while  operating  in  a  TOC  degrades  a  unit’s  ability  to  effectively  plan, 
execute,  and  respond  to  the  battle.  As  a  consequence,  the  ability  to  optimally  train  and  sustain 
digital  proficiency  is  needed  to  maintain  battlefield  readiness.  Common  wisdom  is  that  digital 
skills  are  perishable  yet  little  empirical  evidence  exists  to  document  this  assumption.  Yet,  it  is 
not  enough  to  simply  know  that  digital  skills  are  perishable.  To  plan  effective  training 
schedules,  trainers  and  commanders  must  know  which  skills  and  knowledge  are  perishable  and 
the  types  of  training  methods  that  most  economically  maintain  skill  retention. 

Procedure: 

Thirty-six  Soldiers  from  CPOF  training  at  two  battle  command  training  centers  (BCTC) 
completed  a  skills  test  immediately  following  training  and  again  five  weeks  after  training.  Skill 
retention  on  each  of  the  1 8  CPOF  skills  was  individually  analyzed  for  forgetting  across  the 
retention  interval.  Retention  perfonnance  on  each  skill  was  also  aggregated  across  CPOF-skill 
categories,  i.e.,  Construct,  Visualize,  and  Collaborate.  Self-report  questionnaires  were  used  to 
detennine  Soldiers’  levels  of  general  computer  experience,  military  experience,  and  experience 
with  other  Army  digital  systems  that  might  influence  CPOF-skill  retention. 

Findings: 

Overall,  Soldiers  showed  statistically  significant  forgetting  of  CPOF  skills  from  the  initial 
exercise  to  the  retention  exercise  even  though  the  retention  was  fairly  high  after  a  five-week 
retention  period.  The  results  further  showed  that  retention  loss  was  limited  to  9  of  the  18  CPOF 
skills  analyzed.  Likewise,  there  was  less  retention  of  Collaboration  skills  than  the  other  skill 
categories. 

Utilization  and  Dissemination  of  Findings: 

While  the  skill -retention  results  did  not  specifically  indicate  a  methodology  for 
improving  CPOF  training,  the  pattern  of  retention  was  used  to  identify  the  specific  skills  and  the 
progression  of  skills  that  are  critical  in  developing  CPOF  expertise.  The  results  further  suggested 
that  training  techniques  that  leverage  the  execution  of  sub-goals  and  that  illustrate  overlapping 
CPOF  procedures  should  most  efficiently  train  CPOF  skills.  The  results  and  the  training 
implications  for  this  research  were  briefed  to  BCTC  leadership  and  CPOF  trainers  at  three  Army 
posts. 
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RETENTION  OF  DIGITAL  SKILLS:  COMMAND  POST  OF  THE  FUTURE 


Introduction 

As  the  U.S.  Army  continues  to  provide  Soldiers  with  the  next  generations  of  digital 
systems,  the  digital  skills  needed  to  employ  these  systems  is  evolving  and  may  require  the 
acquisition  and  retention  of  new  skills.  This  is  especially  true  for  the  digital  systems  used  in 
tactical  operations  centers  (TOC)  and  command  posts.  A  Soldier’s  inability  to  utilize  previously- 
learned  digital  skills  while  operating  in  a  TOC  degrades  a  unit’s  ability  to  effectively  plan, 
execute,  and  respond  to  the  battle.  As  new  digital  systems  become  more  complex,  there  is  a  need 
for  empirical  evidence  to  identify  critical  digital  skills  and  to  determine  an  ideal  training 
environment  to  enhance  digital-skill  retention.  The  present  report  provides  data  on  the  retention 
of  digital  skills  for  Command  Post  of  the  Future  (CPOF),  which  can  be  considered  the  vanguard 
of  the  next  generation  of  TOC  digital  systems. 

CPOF  is  a  dynamic  visualization  tool  that  supports  decision  making  in  a  collaborative 
environment.  CPOF  can  assist  in  planning  tactical  operations,  in  tracking  battlefield  operations, 
and  in  providing  update  briefings  to  leaders.  To  accomplish  these  functions,  the  CPOF  interface 
uses  a  customizable  workspace  that  is  based  on  the  user’s  needs  rather  than  a  static  data  format. 
An  example  of  the  CPOF  workspace  is  given  in  Figure  1.  Successful  application  of  CPOF 
requires  the  user  to  decide  which  functions  will  best  address  a  problem  or  need.  While  such  an 
approach  to  digital-systems  design  offers  more  flexibility  and  generality  of  use,  the  flexibility  of 
CPOF  may  also  increase  the  complexity  of  learning  the  interface  and  thus  present  difficulty  for 
digital-skill  retention.  Because  the  CPOF  interface  is  mostly  non-linear  (i.e.,  interaction  with  the 
system  is  not  based  on  prescribed  or  hierarchical  sequences  of  steps  and  data),  there  is  less 
internal  cuing  in  the  interface.  Likewise,  in  non-linear  interfaces,  there  are  no  indicators  that  a 
given  task  has  been  completed.  As  a  consequence,  proficiency  with  non-linear  interfaces  like 
CPOF  requires  a  higher  level  of  understanding  of  task  goals  and  interface  capabilities  (Farrell  & 
Moore,  2000). 


Figure  1.  An  example  of  CPOF  workspace  with  various  products  displayed. 
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The  complex  and  non-linear  fonnat  of  the  CPOF  interface  makes  the  training  and 
retention  of  CPOF  skills  particularly  challenging  (Catrambone,  Wampler,  &  Bink,  2009).  The 
difficulty  in  learning  and  retaining  CPOF  skills  is  likely  based  on  the  fact  that  the  interface  does 
not  support  and  novices  do  not  have  the  organization  of  knowledge  necessary  to  intuitively 
interact  with  the  system.  What  is  more,  there  are  few  opportunities  for  individuals  to  practice 
CPOF  skills  because  systems  are  generally  only  available  in  theater  or  in  staff  exercises.  Thus,  it 
was  the  goal  of  the  present  research  to  understand  the  pattern  of  CPOF  digital  skills  retention  in 
order  to  inform  training. 

By  one  view,  the  purpose  of  training  is  to  execute  skills  in  the  appropriate  context,  and, 
oftentimes,  the  acquisition  is  separated  from  the  execution  context  by  an  extended  period  of  non¬ 
use  (Arthur,  Bennett,  Stanush,  &  McNelly,  1998;  Naylor  &  Briggs,  1961;  Sauer,  Hockey,  & 
Wastell,  2000).  As  a  result,  the  sin  qua  non  of  effective  training  is  the  ability  to  retain  skills  over 
a  period  of  non-use  (i.e.,  a  retention  interval).  In  general,  digital  skills  are  perishable,  and  as  a 
consequence,  digital-skills  training  methods  should  be  sensitive  to  patterns  of  skill  retention 
(Goodwin,  2006;  Goodwin,  Leibrecht,  Wampler,  Livingston,  &  Dyer,  2007).  For  example,  it  has 
been  shown  that  a  given  training  method  may  support  short-term  retention  of  a  skill  but  not 
support  long-tenn  retention  (Sauer  et  ah).  Thus,  identifying  the  more-perishable  digital  skills 
may  help  optimize  digital-skill  training  methods. 

While  there  appear  to  be  a  number  of  factors  that  influence  skill  retention,  the  interaction 
among  skill  characteristics  (e.g.,  complex  vs.  simple  skill),  level  of  original  learning  (e.g., 
overlearning,  method  of  training,  etc.),  and  retention-interval  characteristics  (e.g.,  duration, 
access  to  practice,  etc.)  is  most  crucial  for  detennining  the  level  of  skill  retention  (Arthur  et  al. 
1998;  Hagman  &  Rose,  1983;  Lance  et  al.,  1998;  Naylor,  Briggs  &  Reed,  1962).  Obviously,  if  a 
skill  is  complex,  was  poorly  learned,  and  has  a  long  retention  interval,  the  skill  will  not  be  well 
retained.  Digital  skills  are  generally  considered  complex  cognitive  skills  because  the  purpose  of 
these  skills  is  the  application  of  the  digital  system  to  solve  a  problem  in  a  given  context.  Even 
though  the  interactions  with  digital-system  interfaces  are  largely  procedural,  hierarchical 
knowledge  of  the  system  is  required  to  effectively  navigate  digital  interfaces  and  to  optimize 
problem  solving  (Catrambone  et  al.,  2009;  Davis  &  Yi,  2004).  Because  digital  skills  require 
complex  knowledge,  digital  skills  should  be  susceptible  to  skill-retention  loss. 

Training  battle  staff  on  TOC  digital  systems,  including  CPOF,  generally  occurs  in  some 
combination  of  an  8  -  24-hour  familiarization  course,  a  15  -  40-hour  operators  course,  and  on 
the  job  training  (cf.  Catrambone  et  al.,  2009;  Goodwin,  2006;  Sanders,  1999;  Schaab  &  Moses, 
2001).  Classroom  training  methods  may  vary  across  digital  systems  but  mostly  include  a 
combination  of  instruction,  procedural  modeling,  and  practical  exercises  (see  Goodwin  et  al., 
2007;  Schaab  &  Moses).  Procedural  modeling  (i.e.,  trainees  watch  an  expert  perfonn  a  function 
or  routine  and  then  perform  the  function  themselves)  has  been  shown  to  be  effective  in  training 
digital  skills  (see  Davis  &  Yi,  2004),  but  the  efficacy  of  this  method  for  training  Army  digital 
systems  (within  the  context  of  classroom  training)  is  in  question.  For  example,  training  Force 
XXI  Battle  Command  Brigade  and  Below  (FBCB2)  and  its  predecessor  inter-vehicular 
information  system  were  based  on  the  procedural  modeling  method,  but  the  skills  on  these  digital 
platforms  have  been  shown  to  be  quite  perishable  (Goodwin  et  al.  2007;  Goodwin,  Tucker, 
Wampler,  Gesselman,  &  Johnson,  under  review;  Sanders).  So,  again,  it  may  be  that  even  with 
effective  training  methods,  skill  retention  for  CPOF  may  be  problematic. 
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Another  common  characteristic  of  TOC  digital-systems  training  is  that  there  is  a 
significant  gap  in  time  between  skills  training  and  skills  use  in  an  active  battle  staff  assignment. 
In  that  retention  interval,  Soldiers  do  not  have  access  to  systems  on  which  to  practice  (Goodwin 
et  ah,  2007;  Schaab  &  Moses,  2001).  Although  there  are  no  empirical  estimates  of  the  fonnal 
retention  interval  for  digital-skills  training,  the  research  evidence  suggests  that  there  is 
significant  skill-retention  loss  after  as  little  as  four  weeks  (Goodwin  et  ah;  cf.  Schaab  &  Moses). 
In  the  case  of  CPOF,  Soldiers  who  complete  familiarization  training  or  operator  training 
typically  do  not  have  the  opportunity  to  use  the  system  until  a  staff  exercise  or  deployment, 
which  can  be  several  months  after  training.  As  a  consequence,  the  retention  interval  for  CPOF 
skills  will  likely  lead  to  significant  skill  loss. 

In  order  to  better  understand  the  retention  properties  of  CPOF  skills,  it  is  necessary  to 
define  CPOF  digital  skills.  Even  though  digital  skills  are  generally  thought  to  be  complex 
cognitive  skills,  a  “skill”  can  vary  from  being  able  to  power-up  the  system  to  being  able  to 
communicate  orders  in  a  combat  mission  (see  Bink,  Wampler,  Goodwin,  &  Dyer,  2009).  One 
recent  attempt  to  define  the  nature  of  CPOF  skills  utilized  an  execution-based  knowledge 
elicitation  process  to  identify  “critical”  CPOF  skills  and  the  interdependencies  of  the  skills 
(Catrambone  et  al.,  2009).  Catrambone  et  al.  identified  approximately  50  unique  CPOF  skills  and 
defined  the  sub-goals  associated  with  the  execution  of  each  of  those  skills.  What  is  more, 
Catrambone  et  al.  organized  the  CPOF  skills  into  four  hierarchical  categories  based  on  both 
relations  of  the  individual  skills  and  the  functional  characteristics  of  the  CPOF  system  (i.e.,  the 
construction,  display,  and  sharing  of  tactical  “products”  within  CPOF).  The  organization  of  the 
hierarchy  emphasized  the  main  functional  components  of  CPOF  with  its  military  application 
implicit  within  the  categories.  Accordingly,  the  CPOF  system  is  characterized  by  four  main 
functional  groupings  of  skills:  Construct,  Visualize,  and  Collaborate,  and  System  Basics. 

“System  Basics”  are  the  most  basic-level  skills  and  refer  to  the  tools  and  processes 
needed  for  system  operation  and  the  system  interface  (e.g.,  how  to  properly  apply  mouse  clicks). 
“Construct”  skills  are  also  basic-level  skills  and  are  used  to  construct  simple  products  by 
applying  the  basic  system  tools  (e.g.,  creating  a  map  from  the  Frame  Dispenser).  “Visualize” 
and  “Collaborate”  skills  are  the  highest-level  skills,  are  interdependent,  and  allow  for  the 
application  of  CPOF  in  a  tactically-relevant  manner.  These  higher-level  CPOF  skills  allow  the 
user  to  visualize  current  and  past  battlespace  information  (e.g.,  creating  a  Pasteboard)  and  to 
interactively  share  that  infonnation  with  other  decision  makers  (e.g.,  importing  a  product  to  the 
PASS  Tree). 

The  present  research  tracked  the  retention  of  certain  CPOF  skills  identified  by 
Catrambone  et  al.  (2009).  The  curious  reader  is  invited  to  review  the  Critical  Skills  Document 
contained  in  Catrambone  et  al.  for  a  more  thorough  description  of  each  skill.  It  is  important  to 
note  that  in  both  Catrambone  et  al.  and  the  present  report,  the  consideration  of  CPOF  skills  did 
not  focus  on  the  skills  used  for  the  “3 -dimensional”  display.  The  CPOF  3 -dimensional  display 
includes  skills  specific  to  that  display,  and  because  of  bandwidth  issues,  the  3 -dimensional 
display  is  sparingly  used  in  most  TOC  applications.  Thus,  the  focus  of  skill  retention  reported 
here  only  addresses  skills  used  for  the  “main”  (i.e.,  2-dimensional)  CPOF  display. 
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Given  the  complex  nature  of  digital  skills  in  general,  the  type  of  typical  CPOF  training, 
and  the  CPOF  skill-retention  intervals,  CPOF  skills  should  show  significant  loss  of  retention. 
However,  it  is  not  clear  if  all  CPOF  skills  will  have  the  same  rates  of  retention  loss.  Because  of 
the  hierarchy  of  CPOF  skills,  it  may  be  the  case  that  certain  types  of  CPOF  skills  are  more 
susceptible  to  forgetting  than  others.  For  example,  the  Construct  skills  contain  a  common  set  of 
sub-goals  that  are  reinforced  in  training.  The  procedural  nature  of  the  Construct  skills  and  the 
common  sub-goals  across  these  skills  suggest  that  retention  should  be  robust  (ref.  Catrambone  et 
al.,  2009;  Lance  et  al.,  1998).  By  contrast,  the  higher-level  skill  types,  i.e.,  Visualize  and 
Collaborate,  require  more  system  knowledge  as  well  as  a  certain  degree  of  military-operations 
knowledge.  The  requisite  system  knowledge  for  higher-level  skills  is  not  easily  acquired  (Davis 
&  Yi,  2004;  Dorsey,  Campbell,  Foster,  &  Miles,  1999)  and  may  require  different  training 
methods  than  procedural  skills  (Catrambone,  1998;  Ericsson,  Krampe,  &  Tesch-Romer,  1993). 
As  a  result,  there  will  likely  be  more  forgetting  for  the  higher-level  CPOF  skills  than  for  the 
lower-level  skills.  As  will  be  seen,  the  results  reported  here  will  provide  evidence  to  test  this 
prediction. 

The  research  reported  here  was  designed  to  answer  three  basic  questions:  to  what  degree 
are  CPOF  skills  susceptible  to  retention  loss;  if  there  is  significant  loss  of  retention,  which  CPOF 
skills  show  greater  retention  loss;  and  what  does  the  pattern  of  skill  retention  loss,  if  any,  indicate 
about  potential  training  approaches  for  CPOF.  To  address  these  questions,  skill-performance  data 
was  collected  from  Soldiers  completing  24  hours  of  CPOF  training.  The  Soldiers  had  little  prior 
experience  with  CPOF  and  varied  in  both  digital-system  experience  and  military  experience. 

The  present  research  effort  compliments  recent  FBCB2-skill  retention  research  efforts  as  a 
means  to  optimize  training  of  digital  skills  (see  Goodwin,  et  al.,  2007;  Goodwin,  et  al.,  under 
review).  For  example,  in  a  comparison  of  FBCB2-skill  retention  (Goodwin,  et  al.,  under 
review),  it  was  shown  that  there  are  some  key  steps  on  some  procedures  that  should  be 
emphasized  during  training.  In  addition,  the  patterns  of  skill  retention  differed  between  Soldiers 
in  distance  learning  classes  and  Soldiers  in  traditional  classes,  which  suggested  that  some 
characteristics  of  the  distance-learning  environment  may  be  effective  for  training  digital  skills. 
While  it  was  not  the  intent  of  the  current  research  to  compare  retention  rates  across  different 
training  environments,  it  is  not  unreasonable  to  expect  that  any  difference  in  the  patterns  of 
CPOF-skill  retention  could  indicate  what  types  of  training  might  enhance  retention. 


Method 


Participants 

Thirty-six  Soldiers  from  CPOF  training  at  two  Battle  Command  Training  Centers 
(BCTC)  completed  a  skills  test  immediately  following  training  and  again  about  five  weeks  after 
training.  The  Soldiers  ranged  in  rank  from  Private  First-Class  to  Command  Sergeant  Major  and 
Captain  and  ranged  in  time-in-service  from  15  months  to  324  months  (i.e.,  27  years).  In  general, 
the  Soldiers  were  Specialists  or  junior  non-commissioned  officers  with  less  than  60  months  time- 
in-service.  Most  Soldiers  had  some  TOC  experience.  Six  Soldiers  were  unable  to  successfully 
complete  the  retention  test  because  of  duty  requirements.  As  a  consequence,  the  reported 
analyses  were  based  on  a  total  sample  of  30  Soldiers. 
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Materials  and  Procedure 


All  Soldiers  completed  24-hour  CPOF  training  at  a  BCTC.  As  part  of  the  training  course, 
Soldiers  completed  an  end-of-course  practical  exercise  (i.e.,  initial  practical  exercise).  The  initial 
practical  exercise  took  over  three  hours  for  Soldiers  to  complete  (M  =  3.34hr.,  SEM  =  .24). 
Soldiers  then  returned  to  the  BCTC  four  to  six  weeks  later  depending  on  duty  schedule.  The 
mean  retention  interval  was  37  days  (SEM  =  1.16).  Upon  return  to  the  BCTC,  each  Soldier 
completed  a  second  practical  exercise  to  assess  the  retention  of  CPOF  skills  (i.e.,  retention 
practical  exercise).  Soldiers  also  required  over  three  hours  to  complete  the  retention  exercise  (M 
=  3.68hr.,  SEM  =  .28)  and  were  debriefed  at  the  conclusion  of  the  exercise. 

The  initial  practical  exercise  required  each  Soldier  to  apply  skills  learned  in  training  by 
preparing  CPOF  overlays  and  products  that  might  be  used  for  a  battle-update  brief.  The 
retention  practical  exercise  was  similar,  though  not  identical,  in  format  and  in  content  as  the 
initial  classroom  practical  exercise.  That  is,  the  same  skills  were  utilized  in  each  practical 
exercise  although  the  specific  infonnation  used  to  complete  the  two  practical  exercises  differed. 
For  example,  the  initial  exercise  required  Soldiers  to  create  a  product  for  a  forward  operating 
base  near  Samarra  and  name  the  product  “FOB  Danbury”  while  the  retention  practical  exercise 
required  Soldiers  to  make  a  product  for  a  forward  operating  base  near  Baghdad  and  name  the 
product  “FOB  Apache.”  Appendix  A  provides  the  items  from  the  retention  practical  exercise  as 
an  example  of  the  types  of  items  that  were  used  on  both  practical  exercises.1  As  can  be  seen  in 
Appendix  A,  successfully  accomplishing  a  given  practical-exercise  item  may  have  required  the 
Soldier  to  correctly  apply  different  and  multiple  CPOF  skills.  It  is  important  to  note  that  there 
were  multiple  instances  of  a  given  skill  (e.g.,  “Create  a  Unit”)  on  the  practical  exercises. 

At  the  completion  of  each  practical  exercise,  a  course  instructor  reviewed  each  Soldier’s 
practical  exercise  on  the  CPOF  system  and  noted  on  a  checksheet  whether  each  item  was 
successfully  completed.  The  checksheet  listed  each  specific  CPOF  task  component  required  to 
complete  the  practical-exercise  items,  and  the  Soldier  was  given  a  “Go”  for  successful 
completion  of  the  task  component  or  a  “NoGo”  if  the  task  component  was  not  successfully 
completed.  It  is  important  to  note  that  the  task  components  were  associated  with  specific  CPOF 
skills  identified  by  Catrambone  et  al.  (2009).  Thus,  it  was  possible  to  quantify  each  Soldier’s 
skill  proficiency  on  18  specific  CPOF  skills.  Appendix  B  provides  a  copy  of  the  retention 
practical  exercise  checksheet.  The  copy  of  the  checksheet  indicates  which  CPOF  skill  was 
associated  with  each  practical-exercise  task  component.  It  is  important  to  note  that  the  same 
instructor  at  both  BCTCs  reviewed  both  practical  exercises  (i.e.,  initial  and  retention).  As  a 
result,  there  were  no  issues  with  inter-rater  reliabilities. 

Soldiers  also  completed  a  brief  demographic  questionnaire  at  the  conclusion  of  the  initial 
practical  exercise.  The  questionnaire  contained  items  about  the  Soldier’s  general  military 
experience  (e.g.,  time  in  grade)  and  experience  with  relevant  digital  systems  (e.g.,  FBCB2).  A 
copy  of  the  demographic  questionnaire  is  provided  in  Appendix  C.  At  the  completion  of  the 
retention  practical  exercise,  Soldiers  completed  a  second  questionnaire.  This  questionnaire  was 
designed  to  assess  the  degree  to  which  the  Soldiers  had  any  digital-system  experience  or  training 


1  Because  the  initial  (i.e.,  end-of-course)  practical  exercise  is  current  training  material,  a  copy  of  that  practical 
exercise  was  not  included  in  this  report  in  order  to  maintain  the  proprietary  nature  of  the  material. 
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in  the  retention  interval  that  could  affect  CPOF  skill  retention.  Most  importantly,  the  retention 
questionnaire  was  used  to  determine  if  Soldiers  had  access  to  CPOF  during  the  retention  interval. 
Appendix  D  provides  a  copy  of  the  retention  questionnaire. 


Results 

Throughout  this  paper,  statistical  significance  was  based  on  the  five-percent  level  of 
alpha  error.  The  pair-wise  comparisons  of  means  were  analyzed  with  one -tail  tests  because  only 
decreases  in  perfonnance  values  were  of  interest.  That  is,  the  main  purpose  for  the  analyses  was 
to  identify  CPOF  skills  that  were  not  retained  (i.e.,  statistically  significant  lower  perfonnance  on 
the  retention  exercise  than  the  initial  exercise).  If  a  skill  was  retained,  it  was  of  no  of 
consequence  to  these  analyses  if  the  skill  increased  or  stayed  the  same  across  the  retention 
interval.  Post -hoc  differences  in  means  were  determined  by  pair-wise  comparisons  of  95% 
confidence  intervals.  Where  appropriate,  group  means  and  standard  errors  of  the  mean  are  given 
in  the  text. 

Individual  CPOF  Skills 

Individual  items  from  each  practical  exercise  were  aggregated  according  to  the  individual 
CPOF  skills  represented  in  order  to  allow  comparisons  across  exercises  (refer  to  Appendix  B). 
The  proportion  correct  (i.e.,  rated  a  “Go”)  of  all  items  within  each  skill  was  calculated  for  both 
the  initial  practical  exercise  and  the  retention  practical  exercise.  Comparisons  (i.e.,  paired  t- 
tests)  were  perfonned  on  the  proportions  from  each  exercise  for  each  skill. 

In  general,  retention  of  CPOF  skills  was  fairly  good  over  the  retention  interval.  When 
aggregated  across  all  CPOF  skills  assessed,  Soldiers  showed  statistically  significant  forgetting  of 
CPOF  skills  from  the  initial  exercise  to  the  retention  exercise  (t( 29)  =  2.81,  MSE  =  .037). 
However,  the  proportion  of  correctly  executed  CPOF  skills  only  decreased  by  about  11%  from 
the  initial  exercise  (M=  .94,  SEM=  .02)  to  the  retention  exercise  (M=  .83,  SEM=  .04),  and  the 
proportions  were  fairly  high  even  after  a  five-week  retention  period. 

Each  of  the  1 8  CPOF  skills  assessed  in  the  practical  exercises  was  individually  analyzed 
for  forgetting  across  the  retention  interval.  As  shown  in  Table  1,  nine  of  the  CPOF  skills  showed 
no  statistically  significant  forgetting  (i.e.,  the  skills  were  “retained”).  Likewise,  nine  of  the  CPOF 
skills  showed  a  statistically  significant  decrease  in  proportion  correct  across  the  initial  exercise 
and  the  retention  exercise  (i.e.,  the  skills  were  “not  retained”).  The  largest  difference  between 
proportion  correct  across  initial  and  retention  exercises  was  for  “PASS:  import  &  display 
product,”  (f( 29)  =  3.03,  MSE  =  .068)  which  showed  about  a  20%  decrease  in  perfonnance  (initial 
mean  =  .94,  SEM=  .03;  retention  mean  =  .74,  SEM=  .07). 
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Table  1. 

Mean  Proportions  of  Correct  Responses  for  CPOF  Skills  Retained  and  Not  Retained. 


Initial 

Retention 

CPOF  Skill 

CPOF  Skills 

Exercise 

Exercise 

Category 

Set  Automatic  Layout 

.96  (.03) 

.88  (.05) 

Construct 

Create  Graphic 

.97  (.01) 

.94  (.03) 

Construct 

Name/Label  Graphic 

.99  (.01) 

.95  (.03) 

Construct 

Set  Event  Table  Properties 

.90  (.06) 

.82  (.07) 

Construct 

Retained 

Create  and  Name  Pasteboard 

.96  (.02) 

.93  (.03) 

Visualize 

Create  Master  Schedule 

.93  (.05) 

.97  (.03) 

Visualize 

Create,  Name,  &  Nest  Map 

.97  (.02) 

.90  (.04) 

Visualize 

Create,  Name,  &  Place  Effort 

.99  (.01) 

.92  (.04) 

Visualize 

Set  Privileges 

.85  (.06) 

.73  (.08) 

Collaborate 

Locate  Product 

.99  (.01) 

.90  (.03) 

Construct 

Populate  Effort  (Clone  product) 

.98  (.01) 

.92  (.04) 

Construct 

Create  Unit 

.99  (.01) 

.95  (.03) 

Construct 

Not 

Retained 

Create  Event 

.95  (.04) 

.80  (.07) 

Construct 

Set  Event  Properties 

.97  (.03) 

.83  (.07) 

Construct 

Create  Stickie 

.90  (.05) 

.72  (.08) 

Construct 

Create  &  Name  Map  Preset 

.99  (.01) 

.88  (.05) 

Visualize 

Set  Preset  View 

.91  (.03) 

.81  (.06) 

Visualize 

PASS:  import  &  display  product 

.94  (.03) 

.74  (.07) 

Collaborate 

Notes.  Standard  errors  of  the  mean  are  given  in  parentheses.  “Not  Retained”  was  defined  as  no 
statistical  difference  between  means. 

CPOF  Skill  Categories 

Retention  perfonnance  on  each  skill  was  also  aggregated  across  skill  categories  identified 
by  Catrambone,  et  al.  (2009),  i.e.,  Construct,  Visualize,  and  Collaborate.  In  order  to  specifically 
understand  how  CPOF-skill  retention  varied  as  a  function  of  the  types  of  skills,  proportions  of 
correct  responses  for  the  practical  exercises  were  compared  across  CPOF  skill  categories.  The 
resulting  analysis  yielded  a  statistically  significant  interaction  between  skill  retention  and  skill 
category  (F( 2,  58)  =  3.22,  MSB  =  .019).  Figure  2  displays  the  nature  of  the  interaction. 
Accordingly,  there  was  a  statistically-significant  lower  proportion  correct  for  the  retention 
exercise  (M=  .83,  SEM=  .04)  than  for  the  initial  exercise  (M=  .93,  SEM=  .04)  regardless  of 
skill  category  (F(l,  29)  =  7. 1 1,  MSE  =  .068).  However,  there  was  a  larger  difference  between 
proportions  correct  for  Collaborate  than  either  Construct  or  Visualize.  Thus,  there  was  less 
retention  of  Collaboration  skills  than  the  other  skill  categories. 
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Figure  2.  CPOF  skill  retention  as  a  function  of  skill  categories.  Error  bars  represent  95% 
confidence  intervals. 

In  addition  to  the  patterns  of  retention,  the  correlations  of  difference  scores  among  the 
CPOF-skill  categories  indicated  that  Collaborate  skills  did  not  share  the  retention  properties  of 
the  other  skills.  Difference  scores  for  the  CPOF-skill  categories  were  derived  by  subtracting  the 
proportion  of  correct  responses  on  the  initial  practical  exercise  from  the  proportion  correct 
responses  on  the  retention  exercise  for  each  skill  and  then  aggregating  the  difference  scores 
across  skill  categories.  While  difference  scores  of  Construct  skills  and  Visualize  skills  were 
highly  correlated  (r  =  .84),  Difference  scores  for  Collaborate  skills  were  only  moderately 
correlated  with  both  Construct  (r  =  .55)  and  Visualize  (r  =  .56).  Thus,  it  appeared  that  Visualize 
and  Construct  skills  shared  characteristics  that  likely  contributed  to  the  retention  of  the  skills. 
Those  shared  characteristics  are  most  likely  the  sub-goals  of  the  skills,  but  Visualize  and 
Construct  skills  may  also  have  shared  training  overlap.  That  is,  the  training  program  of 
instruction  for  the  CPOF  courses  focused  heavily  on  Visualize  skills,  and  as  a  consequence,  the 
Construct  skills  were  introduced  in  training  as  a  means  to  develop  Visualize  products. 

It  was  predicted  that  Construct  skills  should  be  better  retained  than  either  Visualize  skills 
or  Collaborate  skills  because  Construct  skills  have  more  common  sub-goals  and  are  precursors 
for  the  higher-level  skills.  The  data  did  not  support  this  prediction  because  the  retention  of 
Construct  skills  and  Visualize  skills  shared  retention  properties  (i.e.,  levels  of  retention  and 
correlations  between  retention  perfonnances)  and  both  differed  from  Collaborate  skills.  In 
addition,  in  reconsidering  the  pattern  of  skill  retention  for  the  individual  CPOF  skills  listed  in 
Table  1,  it  is  important  to  note  that  skills  from  each  CPOF-skill  category  were  both  retained  and 
not  retained  but  that  Visualize  skills  had  the  greater  proportion  of  retained  skills.  That  is,  two- 
thirds  of  the  Visualize  skills  were  retained  while  only  about  half  of  both  Construct  skills  and 
Collaborate  skills  were  retained.  It  is  also  important  to  note  that  the  retention  of  individual 
CPOF  skills  varied  more  as  a  function  of  skill  sub-goals  than  of  the  type  of  skills.  That  is, 
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creating  a  map  was  a  skill  that  was  retained,  but  creating  an  event  was  not  retained.  The  main 
difference  between  these  skills  is  that  creating  an  event  involves  the  additional  sub-goals  of 
providing  specific  infonnation  in  specific  fields  of  the  product  in  addition  to  creating  the  product 
form  the  frame  dispenser  and  locating  the  product  on  a  map.  The  sub-goals  of  providing  specific 
information  in  the  product  are  shared  by  most  of  the  other  “not  retained”  skills  but  are  not 
required  by  the  “retained”  skills.  This  difference  between  the  retained  skills  and  not-retained 
skills  is  important  because  training  should  take  advantage  of  sub-goal  overlap  to  develop  skills 
(Catrambone,  1998;  Catrambone,  et  al.  2009). 

Additional  Factors  Influencing  Skill  Retention 

On  the  demographic  questionnaire  that  each  Soldier  completed,  there  were  a  number  of 
items  that  assessed  the  amount  of  training  and  experience  the  Soldier  had  with  general  computer 
skills  (e.g.,  Microsoft  applications)  as  well  as  with  specific  Army  digital  systems  (e.g.,  FBCB2 
and  All  Source  Analysis  System).  In  addition,  the  demographic  questionnaire  assessed  the  level 
of  military  experience  each  Soldier  had,  such  as  time  in  service,  the  number  of  professional 
military  courses,  and  time  deployed.  The  individual  questionnaire  items  were  aggregated  into 
composite  indicators  of  general  computer  experience,  military  experience,  and  experience  with 
two  individual  Anny  digital  systems  (i.e.,  FBCB2  and  CPOF).  For  example,  military  experience 
indicator  was  created  by  aggregating  responses  from  time-in-service,  amount  of  staff  experience, 
and  number  of  professional  military  courses.  Likewise,  experience  with  Army  digital  systems, 
including  CPOF,  was  a  composite  of  responses  from  items  of  individual  and  unit  training, 
deployment  usage,  and  self-rated  proficiency  with  each  system. 

Each  of  these  dimensions  (i.e.,  general  computer  experience,  military  experience,  and 
individual  Army  digital  systems  experience)  was  correlated  with  retention  perfonnance  for  the 
three  CPOF-skill  categories  (i.e.,  Construct,  Visualize,  and  Collaborate)  as  well  as  with  overall 
retention  performance.  None  of  the  correlations  between  the  CPOF-skill  categories  and  military 
experience  were  statistically  significant  (highest  r  =  .18).  Likewise,  there  were  no  statistically 
significant  correlations  between  CPOF-skill  categories  and  either  FBCB2  experience  (highest  r  = 
.29)  or  prior  CPOF  experience  (highest  r  =  .28).  By  contrast,  the  level  of  general  computer 
experience  statistically  correlated  with  overall  retention  (r  =  .42),  retention  of  Construct  skills  (r 
=  .36),  and  Visualize  skills  (r  =  .47),  but  there  was  no  statistically  significant  correlation  between 
general  computer  experience  and  Collaborate  skills  (r  =  .30).  As  a  result,  it  appeared  that  neither 
military  experience  nor  experience  with  specific  Army  digital  systems  contributed  as  much  to 
CPOF-skill  retention  as  did  general  computer  experience. 

One  of  the  design  principles  for  CPOF  was  to  base  the  interface  on  common  personal- 
computer  procedures  such  as  drag-and-drop  and  copy-and-paste.  The  fact  that  general  computer 
experience  correlated  with  CPOF-skill  retention  seemed  to  indicate  this  design  principle  was 
met.  That  is,  the  results  suggested  that  there  was  a  positive  transfer  from  general  computer 
experience  to  CPOF  skills.  However,  these  results  should  be  interpreted  with  caution  because 
the  questionnaire  was  a  simple  experimental  tool  and  not  a  validated  instrument.  A  more 
systematic  investigation  of  the  transfer  of  general  computer  skills  to  CPOF  skill  may, 
nonetheless,  be  warranted. 
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Eight  Soldiers  reported  having  used  CPOF  during  the  retention  interval,  but  seven  of  the 
eight  reported  using  CPOF  only  once.  Even  though  this  level  of  retention-interval  use  was  so 
slight,  a  comparison  of  CPOF-skill  retention  (i.e.,  proportion  correct  across  all  skills)  was 
conducted  between  those  who  used  CPOF  during  the  retention  interval  and  those  who  did  not  use 
CPOF  during  the  retention  interval.  Statistically,  there  was  no  difference  between  the  retention 
of  CPOF  skills  for  those  who  used  the  system  in  the  retention  interval  and  those  who  did  not  (F  < 
1).  Asa  consequence,  the  limited  exposure  to  CPOF  during  the  retention  interval  did  not  aid  the 
retention  of  skills. 


Discussion  and  Recommendations 

In  summary,  even  though  overall  retention  of  CPOF  skills  was  high,  there  were 
differences  in  retention  across  skills.  In  particular,  Collaborate  skills  had  the  largest  decline  in 
retention,  but  skills  in  each  CPOF-skill  category  demonstrated  forgetting.  These  results  carry 
implications  not  only  for  the  general  understanding  of  digital-skill  retention  but  also  for  the 
understanding  of  CPOF-skill  training. 

Digital-skill  Retention 

The  present  results  add  to  the  empirical  evidence  for  the  loss  of  retention  of  digital  skills 
for  Army  digital  systems.  More  specifically,  the  results  showed  that  retention  loss  occurred  for 
the  complex  and  non-linear  interface  over  a  relatively  short  interval  and  with  high  initial 
perfonnance.  The  fact  that  retention  loss  occurred  for  a  non-linear  interface,  such  as  CPOF,  was 
not  surprising.  Non-linear  interfaces  do  not  provide  the  same  degree  of  internal  cuing  as  do 
menu-driven  linear  interfaces  (e.g.,  FBCB2)  and  may  require  discrete  representations  of 
procedural  execution  even  for  related  skills.  Such  was  the  case  for  the  present  results.  For 
example,  creating  an  effort  was  a  retained  skill  while  creating  an  event  was  not  retained.  Both  of 
these  skills  have  a  common  set  of  procedures  (e.g.,  both  are  pulled  from  the  frame  dispenser, 
both  require  text-field  input,  and  both  are  located  on  maps  in  the  same  way),  but  the 
commonality  of  procedures  clearly  did  not  lead  to  performance  transfer  across  skills. 

The  fact  that  the  present  research  had  both  a  relatively  short  five-week  retention  interval 
and  very  high  initial  skill  performance  (i.e.,  M=  .94)  further  suggested  that  digital  skills  are 
particularly  susceptible  to  retention  loss.  Loss  of  digital-skill  retention  has  been  shown  with  as 
little  as  a  30-day  interval  (Sanders,  1999)  while  perfonnance  on  other  complex  skills  showed  no 
retention  loss  after  as  long  as  eight  months  (see  Sauer,  et  ah,  2000).  The  present  results  clearly 
demonstrated  that  digital  skills  can  be  quickly  impaired  if  not  practiced.  Interestingly,  the  loss  of 
retention  for  CPOF  skills  occurred  even  though  perfonnance  was  quite  good  immediately  after 
training.  However,  none  of  the  Soldiers  in  the  present  research  had  the  level  of  training 
previously  shown  to  reduce  skill  loss.  That  is,  the  CPOF  training  used  by  the  BCTCs  did  not 
include  overlearning,  which  is  additional  training  beyond  skill  proficiency  (Farr,  1987;  Schendel, 
Shields,  &  Katz,  1978).  In  addition,  there  was  no  relation  between  initial  level  of  perfonnance 
on  CPOF  skills  and  performance  after  the  retention  interval  (r  =  .22).  Thus,  it  was  difficult  to 
detennine  if  the  loss  of  retention  of  CPOF  skills  was  due  to  or  in  spite  of  the  level  of  initial 
perfonnance. 
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Another  implication  for  the  relatively  high  rates  of  initial  perfonnance  was  that  perhaps 
the  CPOF  interface  is  not  as  overly  complex  as  previously  suggested  (Catrambone,  et  ah,  2009; 
Middlebrooks,  2008).  Even  though  CPOF  was  designed  to  fulfill  a  complex  operational  role,  the 
CPOF  interface  was  designed  to  resemble  current  personal-computer  interfaces.  In  fact,  the 
present  results  showed  that,  overall,  the  more  general  computer  experience  Soldiers  had,  the 
more  CPOF  skills  were  retained.  Perhaps,  the  perception  of  complexity  for  CPOF  is  based  more 
on  the  operational  application  of  the  system  rather  than  on  the  skills  used  to  interface  with  the 
system. 


Even  though  the  current  results  support  the  position  that  digital  skills  are  perishable  (i.e., 
Goodwin,  2006),  general  statements  about  the  susceptibility  of  digital  skills  to  retention  loss 
should  be  avoided.  Currently,  there  is  no  consensus  in  the  research  literature  about  the 
perishability  of  digital  skills.  For  example,  no  statistically  significant  loss  of  retention  was  found 
for  All  Source  Analysis  System/Remote  Workstation  skills  in  military  analysts  after  a  three-to- 
four  month  retention  interval  (Schaab  &  Moses,  2001).  Likewise,  both  the  present  results  and 
the  results  of  Goodwin  et  al.  (under  review)  indicated  that,  even  though  there  was  overall  loss  of 
retention,  not  all  individual  digital  skills  were  susceptible  to  retention  loss.  In  the  case  of 
Goodwin  et  al.,  overall  perfonnance  on  multi-step  procedures  showed  retention  loss  whereas 
critical  portions  of  the  multi-step  procedures  showed  no  loss  of  retention.  As  for  the  present 
results,  the  overall  loss  of  retention  was  driven  by  half  of  the  individual  skills  tested.  That  is, 
there  was  statistically  significant  retention  loss  for  nine  of  the  individual  CPOF  skills  and  no  loss 
of  retention  for  the  other  nine  CPOF  skills  tested.  What  is  more,  skill  retention  varied  as  a 
function  of  CPOF-skill  categories  in  the  present  results,  and  there  was  no  consistent  pattern  of 
retention  across  skills  that  shared  sub-goals.  The  lack  of  a  cohesive  pattern  of  results  across  the 
digital-skill  retention  literature  suggested  that  a  more  systematic  investigation  of  digital  skills  is 
needed  before  general  statements  about  the  retention  properties  can  be  made. 

Because  skill  retention  depends  on  the  interaction  of  skill  characteristics,  level  of  original 
learning,  and  retention-interval  characteristics  (Arthur,  et  al.,  1998;  Naylor  &  Briggs,  1961), 
future  research  designed  to  simply  document  the  presence  of  or  lack  of  digital  skill  retention  will 
have  limited  applicability.  Merely  demonstrating  loss  of  digital-skill  retention  cannot  help  guide 
training  unless  other  factors  are  appropriately  considered.  However,  patterns  of  skill  retention 
observed  in  a  given  training  environment  can  be  used  to  determine  a  sequence  of  training.  For 
example,  as  will  be  described,  the  results  from  the  present  research  can  be  used  to  suggest  that 
training  CPOF  skills  that  were  better  retained  should  precede  the  training  of  skills  that  were  not 
retained.  What  is  more,  training  the  better-retained  skills  should  be  reinforced  as  more  focus  is 
given  to  training  the  skills  that  were  not  retained.  Likewise,  the  types  of  training  modalities 
used  to  train  CPOF  skills  can  be  chosen  to  leverage  the  pattern  of  skill  retention. 

Training  CPOF  Skills 

In  general,  sequencing  training  for  complex  skills,  like  digital  systems,  should  begin  with 
skills  that  provide  effective  strategies  (Clawson,  Healy,  Ericsson,  &  Bourne,  2001)  or  with  skills 
that  leverage  common  sub-goals  (Catrambone,  1998).  In  the  case  of  CPOF  skills,  applying  these 
two  general  guidelines  was  a  matter  of  detennining  which  skill  sub-goals  were  common  across 
both  retained  skills  and  skills  not  retained.  The  set  of  common  sub-goals  is  listed  in  Table  2. 
These  skill  sub-goals  represent  the  general  procedural  steps  for  producing  many  of  the  CPOF 


11 


products  and  can  be  generalized  as  new  skills  are  introduced  in  training.  Training  these  specific 
sub-goals  up  front  and  continually  reinforcing  them  as  new  skills  are  trained  should  be  a  way  to 
provide  an  effective  learning  strategy. 

Table  2. 

Order  of  Execution  of  Common  Sub-goals  for  CPOF  Skills. 

CPOF  Skills  Sub-Goals 

Retrieve  item  from  Frame  Dispenser 
Input  product  infonnation  and  Name 

Drag  product  to  desktop _ 

Use  drop  down  boxes  to  select 

features _ 

Drag  to  desired  location  on 
Pasteboard 

Click  "Nesting  Icon"  while  available 


Training  the  common  sub-goals  is  just  the  first  step  in  defining  the  sequence  of  CPOF- 
skills  training.  Based  on  the  retention  properties  of  CPOF  skills  and  the  general  guidance  on 
sequencing  the  progression  of  skills  (i.e.,  Catrambone,  1998;  Clawson,  et  ah,  2001),  a  sequence 
of  skills  training  can  be  defined.  A  set  of  basic  CPOF  skills  should  be  first  introduced  because 
general  system  knowledge  supports  the  execution  of  specific  CPOF  skills  (Catrambone  et  ah, 
2009).  These  basic  skills  represent  the  general  functionality  of  CPOF  and  are  common  across 
the  CPOF-skill  categories.  The  three  most  basic  CPOF  skills  are  using  the  Frame  Dispenser  (i.e., 
the  source  for  creating  most  other  products),  creating  a  Pasteboard  (i.e.,  the  highest  level  product 
and  most  operationally  relevant),  and  creating  a  Map  (i.e.,  the  most  basic  visualization  tool). 

After  these  basic  skills  are  introduced,  a  set  of  skills  that  are  better  retained  and  that  have 
the  sub-goals  listed  in  Table  2  should  be  trained.  Doing  so  will  reinforce  the  basic  sub-goal 
procedures  and  will  introduce  relatively  easy  skills.  Likewise,  this  set  of  skills  should  be 
Construct  skills  because  Construct  skills  support  the  higher-level  skills.  The  next  steps  in  the 
progression  are  to  introduce  more  difficult  (i.e.,  skills  not  retained)  Construct  skills,  introduce 
Visualize  skills  that  were  retained,  and  continue  to  train  retained  and  not-retained  skills  as 
higher-level  skills  are  introduced.  Table  3  displays  a  suggested  progression  of  skills  training  and 
provides  examples  of  specific  CPOF  skills  taken  from  the  current  research  that  apply  to  each 
level  of  the  progression. 

By  following  this  progression,  increasingly  complex  skills  or  skills  that  are  more 
susceptible  to  forgetting  can  be  trained  in  the  context  of  “easier”  skills.  The  specific  skills 
introduced  at  each  step  of  the  sequence  can  be  determined  not  only  by  the  procedural 
commonality  with  already-learned  skills  but  also  by  the  operational  relevance  of  the  skill  (e.g., 
creating  a  unit  in  the  context  of  battle  tracking).  With  these  two  factors  in  mind  (i.e.,  the 
progression  of  skills  and  operational  relevance),  specific  training  problems  can  be  constructed  to 
form  the  basis  of  the  training  approach.  The  problems  should  be  developed  across  the  different 
CPOF  purposes  (i.e.,  battle  tracking,  battle  planning,  and  update  briefing)  and  should  require 
decision  making  and  collaboration  at  every  level. 
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Table  3. 

Suggested  Progression  of  CPOF  Skills  Training. 


Progression  of  Skills 

Example  Skills  to  be  Trained 

Train  some  basic  system  skills. 

Starting  and  Stopping  the  System 

Introduce  basic  CPOF  products. 

Create  Pasteboard;  Create  Map 

Introduce  Retained  Construct  skills,  and 
have  students  apply  that  knowledge  in 
an  operationally-relevant  problem¬ 
solving  task. 

Create  Graphic 

Introduce  Not-Retained  Construct  skills 
and  Retained  Visualize  skills  that 
incorporate  common  sub-goals,  and 
have  students  apply  that  knowledge. 

Create  Unit;  Create  Effort 

Introduce  Not-Retained  Visualize  skills 
and  Retained  Collaborate  skills  that 
incorporate  common  sub-goals,  and 
have  students  apply  that  knowledge. 

Set  Preset  View;  Set  Privileges 

Continue  to  introduce  and  incorporate 
more  complex  skills  while  reinforcing 
the  training  with  application. 

PASS:  Import  &  Share  product 

Even  though  the  proposed  training  progression  is  a  logical  extension  of  the  available  data, 
several  issues  remain  unresolved.  These  issues  may  impact  the  actual  efficiency  of  the  proposed 
progression  and  the  way  in  which  the  proposed  progression  can  be  implemented.  First,  not  all 
important  CPOF  skills  were  tested  for  retention  (e.g.,  Workspace  Management).  Including  these 
untested  skills  would  be  necessary  for  any  new  CPOF-training  approach,  but  not  enough 
information  is  available  at  this  time  to  determine  where  these  important  and  untested  skills  would 
be  introduced  in  the  training  sequence.  Second,  the  retention  properties  of  CPOF  skills  would 
likely  change  as  a  result  of  the  type  of  training  used  (Arthur  et  ah,  1998).  Thus,  in  order  to 
properly  sequence  skills  training,  the  retention  of  CPOF  skills  would  need  to  be  assessed  after 
the  proposed  training  approach  is  implemented,  and  the  sequence  of  training  should  be  modified 
according  to  those  results. 

The  current  results  do  not  prescribe  a  specific  training  modality  for  CPOF  skills. 
However,  research  on  training  other  digital  systems  suggests  that  problem-based  training  was 
effective  for  complex  systems  like  CPOF.  For  example,  in  the  comparison  of  constructivist 
training  techniques  (e.g.,  problem-based  training)  to  lecture-based  training  for  the  All  Source 
Analysis  System  and  the  Advanced  Field  Artillery  Tactical  Data  System,  results  showed  that 
problem-based  training  produced  higher  scores  than  lecture -based  training  on  the  performance- 
based  practical  exercises  at  the  end  of  digital-skills  courses  (Childs,  Blankenbeckler,  &  Dudley, 
2001;  Childs,  Schaab,  &  Blankenbeckler,  2002).  In  addition,  Childs  et  al.  (2002)  reported  that 
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problem-based  training  allowed  for  more  material  to  be  trained  in  less  time  without  the 
perception  of  additional  workload. 

Of  course,  problem-based  techniques  might  be  only  one  tool  for  training  CPOF  skills 
based  on  the  results  presented  in  this  paper.  The  structure  of  the  proposed  training  progression 
suggested  that  a  mix  of  training  techniques  may  be  appropriate.  For  example,  providing  direct 
instruction  on  the  System  Basics  should  precede  problem-based  exercises  in  order  to  provide 
requisite  system  knowledge.  From  that  point,  a  series  of  problems  that  focus  on  simple  sub  ¬ 
goals  and  a  progression  of  skills  could  be  executed.  It  may  also  be  the  case  that  the  progression 
of  skills  training  could  be  accomplished  with  an  effective  technique  such  as  deliberate  practice 
(e.g.,  Ericsson  et  al.,  1993).  However,  the  main  advantage  of  a  problem-solving  approach  is  that 
the  relations  among  skills  can  be  implicitly  trained  without  additional  explicit  training  on  the 
structure  among  skills. 

Conclusion 

Given  the  increasing  complexity  and  non-linearity  of  Anny  digital-systems  interfaces, 
traditional  digital-system  training  approaches  (i.e.,  memorization  of  key  strokes  and  menus)  will 
limit  the  degree  of  training  efficiency.  In  order  to  avail  oneself  of  the  capabilities  of  complex 
non-linear  systems,  the  user  must  know  the  system  capability  and  understand  how  the  system  can 
be  applied  to  meet  operational  needs.  That  is,  the  user  must  not  only  know  how  to  do  things,  but 
also  know  when  to  do  them  (i.e.,  decision  rules).  The  approach  to  training  development  offered 
in  this  paper  specifically  allows  hierarchical  knowledge  of  complex  non-linear  digital  systems  to 
be  trained. 
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Acronyms 


BCTC 

CPOF 

FBCB2 

TOC 


Battle  Command  Training  Center 
Command  Post  of  the  Future 
Force  XXI  Battle  Command  Brigade  and  Below 
Tactical  Operations  Centers 
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APPENDIX  A 

RETENTION  CPOF  PRACTICAL  EXERCISE 


A-l 


Create  a  Pasteboard: 

Pasteboard  Name:  ROLE  NAME  Overlay  PE 
Create  and  Nest  Map. 


A.  Create  the  following  map  Presets 

1.  Go  to  this  grid,  38  SLF  31897  23368,  to  create  a  GeoStickie  on  the  2D  map  in  Mosul.  Zoom  in 
on  the  GeoStickie  to  Imagery  View.  Make  a  Preset,  Name  it  Mosul. 

2.  Go  to  this  grid,  38  SMB  07669  07879,  to  create  a  GeoStickie  on  the  2D  map  in  Karbala.  Zoom 
in  on  the  GeoStickie  to  Imagery  View.  Make  a  Preset,  name  it  Karbala. 

3.  Go  to  this  Grid,  38  SLB  86788  91606.  Make  a  Preset,  name  it  Fallujah. 

B.  Create  DIV/BDE/BN  Boundaries 

1.  Division  Boundary :  Draw  a  Boundary  from  38SMC  17756  03889  to  38SMC  63530  06307  and 
a  separate  Boundary  from  38SMB  18407  72092  to  38  SMB  64239  71375. 

2.  Change  Echelon  to  Division. 

3.  Label  both  sides  of  the  Division  Boundary  in  the  Unique  Designation  block  or  26  Font  Text: 
Northern  Boundary  -  MND-C  on  North  side  &  MND-B  on  the  Baghdad  side. 

Southern  Boundary  -  MND-B  on  the  Baghdad  side  and  MND-S  on  the  South  side. 

4.  Brigade  Boundary  -  From  North  to  South  Division  Boundaries,  follow  the  River  in  the  center  of 
Baghdad  with  a  Brigade  Boundary;  Label  the  brigade  boundary  on  the  west  side  1BCT  in  20 
font.  Label  the  brigade  boundary  on  the  east  side  2BCT  in  20  font.  Be  careful  when  labeling  the 
boundaries  with  the  Unique  Designation. 

5.  Battalion  Boundary.  Draw  Battalion  Boundaries  within  the  1BCT  area  for  1-5  (west)  and  1-8 
(east):  38SMB  18902  72555  to  38SMB  41510  89501.  Draw  Battalion  Boundaries  within  the 
2BCT  area  for  2-5  (west)  and  2-8  (east):  38SMB  44732  87664  to  38SMC  62864  02618. 

6.  Phase  Lines :  Draw  a  phase  line  from  38SMB  15467  88205  to  38SMB  65462  88311.  Color  the 
phase  line  red  and  label  it  PL  RED. 

7.  Drag  an  Effort,  name  it  BOUNDARIES  AND  PHASE  LINES. 

8.  Drag  all  graphics  to  the  Effort  and  drag-drop  the  Effort  to  the  Effort  List. 

C.  Create  Units 

1 .  Create  the  following  units  and  put  them  on  the  grid  listed: 

•  1BCT  -  3 8 SMB  29362  94788  Infantry  Mech 

•  1-5BN  -  3 8 SMB  23144  83844  Infantry  Mech 

•  1-8BN-  38SMB  37291  78122  Annor 

•  2BCT  -  38SMC  42900  00402  Annor 

•  2-5BN  -  3 8 SMB  51253  98290  Annor 

•  2-8BN  -  3 8 SMB  54339  85382  Infantry  Mech 

2.  Drag  an  Effort,  name  it  BDE/BN  Units  and  drag  effort  to  the  effort  list. 
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D.  Manage  the  3D  Map 

1.  Go  to  the  Fallujah  preset,  build  an  Air  Path  along  the  road  starting  at:  38SLC  59149  26437  and 
ending  at:  38SMB  08845  84614;  what  is  the  distance  of  the  Air  Path?  Add  a  Stickie  to  the  map 
at  either  end  of  the  Air  Path  with  the  distance. 

2.  Still  on  the  Fallujah  preset,  create  a  Blue  Assertion/Field  Artillery  Battalion.  Name  it  2-82,  and 
drop  on  map. 

3.  Create  a  Red/Hostile/Kidnapping  event,  name  it  CNN  Reporters,  drop  on  map. 

4.  Create  a  Distance  Tool. 

5.  Save  all  your  Graphics  and  Events  in  an  Effort  and  name  it  3D  Map  PE,  drag  to  the  map. 

E.  Create  Events  and  Edit  Graphics 

1 .  Go  to  the  Mosul  Preset,  add  the  following  events  with  the  attached  event  description: 

•  SAF  -  A  1-5CAV  patrol  was  attached  with  SAF  in  Baghdad  ivo  38SMB  3410  8770.  While 
investigating  suspicious  activity,  the  patrol  was  attacked  with  4  x  rounds  of  SAF.  There  were  no 
injuries  or  damages  reported. 

•  IED  -  A  D/1-9  CAV  patrol  was  attached  with  an  IED  ivo  of  Baghdad  at  38SMB  3830  6040. 
There  was  1  x  WIA  (shrapnel  to  left  leg  and  left  shoulder),  and  1  x  disabled  Ml 025. 

2.  In  Mosul,  add  a  Division  Boundary;  pick  any  road  on  this  preset  and  make  it  an  MSR. 

3.  Still  in  Mosul,  edit  the  MSR  and  make  it  an  ASR. 

4.  Save  all  the  graphics  and  Events  in  an  effort,  name  it  Mosul  Graphics,  drag  to  the  map. 

F.  Pasteboards,  Nesting  and  Layouts 

1 .  Rename  the  current  map  your  [ROLENAME] . 

2.  Drag  two  separate  Pasteboards  with  Maps  nested.  Name  one  pasteboard  [ROLENAME]  BTL 
CPT  WS,  and  the  other  [ROLENAME]  BTL  CPT  COP. 

3.  Nest  these  two  pasteboards  on  your  Overlay  PE  Pasteboard,  drag  a  Master  Schedule,  Events 
Table  and  a  Table,  and  nest  to  the  Overlay  PE  Pasteboard.  Organize  your  pasteboards  in 
Automatic  Layout. 

G.  Privileges 

1 .  Give  Privileges  to  your  BTL  CPT  WS  Map  to  any  User  Role. 

2.  Give  Privileges  to  the  BTL  CPT  COP  Pasteboard  to  any  User  Role. 

H.  Shared  Products 

1 .  Drag  out  a  Shared  Products  from  the  Frame  Dispenser,  drag-drop  the  Blue  SA  Effort  from  the 
Shared  Products  to  your  map;  activate  the  Current  PLI. 

2.  Subscribe  to  the  PASS,  drag-drop  an  MCS  Overlay  called  Annex_C_BDE_FRAGO_01-07  and 
MND_I_ANNEXC  to  the  BTL  CPT  WS  map,  show  on  the  map. 

3.  Subscribe  to  the  PASS,  drag-drop  an  ASAS  overlay  called  2BCT_Ethnicity  on  the  BTL  CPT 
COP  Map  and  show. 

4.  Subscribe  to  the  PASS,  drag-drop  a  BCS3  overlay  called  CORPxANNEXIxMSR  on  the  BTL 
CPT  COP  map  and  show. 

THIS  ENDS  THE  PRACTICAL  EXERCISE 
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APPENDIX  B 

RETENTION  CPOF  PRACTICAL  EXERCISE  CHECKSHEET* 


"The  specific  CPOF  skill  associated  with  each  checksheet  item  is  given  in  the  “Comments” 
column.  On  the  actual  checksheets,  the  Comments  column  was  used  to  describe  why  an  item 
was  a  “NoGo.” 
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CPOF  Skill  Retention  Scoring  Checklist 

Retention  Practical  Exercise 


GOAL 

The  goal  of  the  observation  is  to  document  the  student’s  results  from  the  CPOF  practical 
exercises. 


SCORER’S  DUTIES 

•  Use  the  sign-in  roster  to  determine  the  amount  of  students  available 

•  Provide  1  Demographic  questionnaire,  1  Scoring  Checklist,  and  2  Take  Home  Sheets  per 
Student 

•  Upon  completion  of  the  Practical  Exercise: 

o  verify  Student’s  information 

o  verify  the  Student  has  2  copies  of  the  take  home  document 
o  begin  the  scoring  procedure. 

•  On  the  checklist,  provide  a  comment  for  each  “no”  rating  to  indicate  what  mistake  was 
made. 

•  After  the  event,  organize  all  notes  and  protocols,  then  deliver  them  to  the  Data  Compiler. 

ADMINISTRATIVE  DATA 

1.  ID  Number _ 

2.  Computer  Role  Name _ 

3.  Date  of  Observation _ 

4.  Practical  Exercise  Start  Time  (24  hr  clock) _ 

5.  Practical  Exercise  Stop  Time  (24  hr  clock) _ 
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Taskl  Create  Pasteboard 

Comments 

a 

Pasteboard  Name: 

ROLE  NAME  Overlay  PE 

Go 

No  Go 

Create  and  Name  Pasteboard 

b 

Nested  Map  Name: 

ROLE  NAME 

Go 

No  Go 

Create,  Name,  &  Nest  Map 

Role  Name  Map  -Mosul  Preset 


Task  2  Create  Presets 

Comments 

a 

Label:  Mosul 

Go 

No  Go 

Create  &  Name  Mao  Preset 

b 

Loc:  38SLF  31897  23368 

Go 

No  Go 

Create  &  Name  Mao  Preset 

c 

Zoom  Level:  Imagery 

Go 

No  Go 

Set  Preset  View 

Role  Name  Map  -  Karbala  Preset 


Task  3  Create  Presets 

Comments 

a 

Label:  Karbala 

Go 

No  Go 

Create  &  Name  Mao  Preset 

b 

Loc:  38SMB  07669  07879 

Go 

No  Go 

Create  &  Name  Mao  Preset 

c 

Zoom  Level:  Imagery 

Go 

No  Go 

Set  Preset  View 

Role  Name  Map  -  Falluiah  Preset 


Task  4  Create  Presets 

Comments 

a 

Label:  Fallujah 

Go 

No  Go 

Create  &  Name  Mao  Preset 

b 

Loc:  38SLB  86788  91606 

Go 

No  Go 

Create  &  Name  Mao  Preset 

c 

Zoom  Level:  Non-imagery 

Go 

No  Go 

Set  Preset  View 
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Role  Name  Map  -  Baqhdad/TAJI  Preset 


Task  5  Division  Boundary 

Comments 

a 

Graphic  Control  Measure:  Boundary  (XX) 

Go 

No  Go 

Create  Graphic 

North  Boundary 

b 

Start/End  Point:  38SMC  17756  03889  -  38SMC 
63530  03607 

Go 

No  Go 

Create  Graphic 

c 

Label:  MND-C  on  No  Gorth  side 

Go 

No  Go 

Name/Label  Graphic 

d 

Label:  MND-B  on  Baghdad  side 

Go 

No  Go 

Name/Label  Graphic 

South  Boundary 

e 

Start/End  Point:  38SMB  18407  72092  -  38SMB 
64239  71375 

Go 

No  Go 

Create  Graphic 

f 

Label:  MND-B  on  Baghdad  side 

Go 

No  Go 

Name/Label  Graphic 

g 

Label:  MND-S  on  South  side 

Go 

No  Go 

Name/Label  Graphic 

Task  6  Brigade  Boundary 

Comments 

a 

Graphic  Control  Measure:  Boundary  (X) 

Go 

No  Go 

Create  Graphic 

b 

Start/End  Point:  Followed  River  in  center  of 
Baahdad 

Go 

No  Go 

Create  Graphic 

c 

Label:  1BCT  on  West  side 

Go 

No  Go 

Name/Label  Graphic 

d 

Label:  2BCT  on  East  side 

Go 

No  Go 

Name/Label  Graphic 
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Role  Name  Map  -  Baqhdad/TAJI  Preset 


Task  7  Battalion  Boundary 

Comments 

a 

Graphic  Control  Measure:  Boundary  (II) 

Go 

No  Go 

Create  Graphic 

1BCT  Boundary 

b 

Start/End  Point:  38SMB  18902  72555  -  38SMB 
41510  89501 

Go 

No  Go 

Create  Graphic 

c 

Label:  1-5  on  West  side 

Go 

No  Go 

Name/Label  Graphic 

d 

Label:  1-8  on  East  side 

Go 

No  Go 

Name/Label  Graphic 

2BCT  Boundary 

e 

Start/End  Point:  38SMB  44732  87664  -  38SMC 
62864  02618 

Go 

No  Go 

Create  Graphic 

f 

Label:  2-5  on  West  side 

Go 

No  Go 

Name/Label  Graphic 

g 

Label:  2-8  on  East  side 

Go 

No  Go 

Name/Label  Graphic 

Task  8  Phase  Line 

Comments 

a 

Graphic  Control  Measure:  Phase  Line 

Go 

No  Go 

Create  Graphic 

b 

Start/End  Point:  38SMB  15467  88205,  38SMB 

65462  88311 

Go 

No  Go 

Create  Graphic 

c 

Color:  Red 

Go 

No  Go 

Create  Graphic 

d 

Label:  PL  Red  (Check  for  duplicate  label  on  3D 
map,  ex:  PL  PL  Red.) 

Go 

No  Go 

Name/Label  Graphic 
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Role  Name  Map  -  Baqhdad/TAJI  Preset 


Task  9  Create  Units 

Comments 

a 

Label:  1BCT 

Go 

No  Go 

Create  Unit 

b 

Loc:  38SMB  29362  94788 

Go 

No  Go 

Create  Unit 

c 

Type:  Infantry  Mech 

Go 

No  Go 

Create  Unit 

d 

Echelon:  Brigade 

Go 

No  Go 

Create  Unit 

e 

Label:  1-5 

Go 

No  Go 

Create  Unit 

f 

Loc:  38SMB  23144  83843 

Go 

No  Go 

Create  Unit 

g 

Type:  Infantry  Mech 

Go 

No  Go 

Create  Unit 

h 

Echelon:  Battalion 

Go 

No  Go 

Create  Unit 

i 

Label:  1-8 

Go 

No  Go 

Create  Unit 

j 

Loc:  38SMB  37291  78122 

Go 

No  Go 

Create  Unit 

k 

Type:  Armor 

Go 

No  Go 

Create  Unit 

1 

Echelon:  Battalion 

Go 

No  Go 

Create  Unit 

m 

Label:  2  BCT 

Go 

No  Go 

Create  Unit 

n 

Loc:  38SMC  42900  00402 

Go 

No  Go 

Create  Unit 

o 

Type:  Armor 

Go 

No  Go 

Create  Unit 

P 

Echelon:  Brigade 

Go 

No  Go 

Create  Unit 

q 

Label:  2-5 

Go 

No  Go 

Create  Unit 

r 

Loc:  38SMB  51253  98290 

Go 

No  Go 

Create  Unit 

s 

Type:  Armor 

Go 

No  Go 

Create  Unit 

t 

Echelon:  Battalion 

Go 

No  Go 

Create  Unit 

u 

Label:  2-8 

Go 

No  Go 

Create  Unit 

V 

Loc:  38SMB  54339  85382 

Go 

No  Go 

Create  Unit 

w 

Type:  Infantry  Mech 

Go 

No  Go 

Create  Unit 

X 

Echelon:  Battalion 

Go 

No  Go 

Create  Unit 
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Role  Name  Map  -  Baqhdad/TAJI  Preset 


Task  10  Create  Effort 

Comments 

a 

Label:  Boundaries  and  Phase  Lines 

Go 

No  Go 

Create,  Name,&  Place  Effort 

b 

Elements:  Boundaries  &  Phase  lines 

Go 

No  Go 

Create,  Name,&  Place  Effort 

c 

BDE/BN  Units 

Go 

No  Go 

Populate  Effort 

d 

Dropped  in  Effort  List 

Go 

No  Go 

Locate  Product 

Task  11  Create  Effort 

Comments 

a 

Label:  BDE/BN  Units 

Go 

No  Go 

Create.  Name,&  Place  Effort 

b 

Elements:  1BCT 

Go 

No  Go 

Create,  Name,&  Place  Effort 

c 

1-5  BN 

Go 

No  Go 

Populate  Effort 

d 

1-8  BN 

Go 

No  Go 

Populate  Effort 

e 

2BCT 

Go 

No  Go 

Populate  Effort 

f 

2-5  BN 

Go 

No  Go 

Populate  Effort 

g 

2-8  BN 

Go 

No  Go 

Populate  Effort 

h 

Dropped  in  Effort  List 

Go 

No  Go 

Locate  Product 
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Role  Name  Map  -  Falluiah  Preset  on  the  3D  Map 


Task  12  Create  Air  Path 

Comments 

a 

Graphic  Control  Measure:  Air  Path 

Go 

No  Go 

Not  scored  in  retention  test 

b 

Add  Stickie  with  distance 

Go 

No  Go 

Not  scored  in  retention  test 

c 

Distance:  -75km 

Go 

No  Go 

Not  scored  in  retention  test 

Task  13  Create  Blue  Assertion 

Comments 

a 

Blue  Assertion  on  map? 

Go 

No  Go 

Not  scored  in  retention  test 

b 

Label:  2-82 

Go 

No  Go 

Not  scored  in  retention  test 

c 

Type:  Field  Artillery 

Go 

No  Go 

Not  scored  in  retention  test 

d 

Echelon:  Battalion 

Go 

No  Go 

Not  scored  in  retention  test 

Task  14  Create  Red  Event 

Comments 

a 

Red  Event  on  map? 

Go 

No  Go 

Not  scored  in  retention  test 

b 

Label:  Red 

Go 

No  Go 

Not  scored  in  retention  test 

c 

Type:  Kidnapping  CNN  Reporter 

Go 

No  Go 

Not  scored  in  retention  test 

d 

Affiliation:  Hostile 

Go 

No  Go 

Not  scored  in  retention  test 

Task  15  Create  Distance  Tool 

Comments 

a 

Displayed  on  Map 

Go 

No  Go 

Not  scored  in  retention  test 
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Role  Name  Map  -  Falluiah  Preset  on  the  3D  Map 


Task  16  Create  Effort 

Comments 

a 

Label:  3D  Map  PE 

Go 

No  Go 

Create,  Name,&  Place  Effort 

b 

Elements:  Air  Path 

Go 

No  Go 

Create,  Name,&  Place  Effort 

c 

Stickie  w/distance 

Go 

No  Go 

Populate  Effort  /  Create  Stickie 

d 

Blue  Assertion 

Go 

No  Go 

Populate  Effort 

e 

Red  event 

Go 

No  Go 

Populate  Effort 

f 

Distance  Tool 

Go 

No  Go 

Populate  Effort 

g 

Dropped  in  Effort  List 

Go 

No  Go 

Locate  Product 

Role  Name  Map  -  Mosul  Preset 


Task  17  Create  Events 

Comments 

a 

Type:  SAF 

Go 

No  Go 

Create  Event 

b 

Title 

Go 

No  Go 

Create  Event 

c 

Unit:  1-5  CAV  Patrol 

Go 

No  Go 

Create  Event 

d 

Loc:  ivo  38SM  B341  08770 

Go 

No  Go 

Set  Event  Properties 

e 

Comments:  Patrol  was  attacked  w/  4  rounds  SAF. 
No  Go  injuries  and  No  Go  damages  reported. 

Go 

No  Go 

Set  Event  Properties 

f 

Type:  IED 

Go 

No  Go 

Create  Event 

g 

Title 

Go 

No  Go 

Create  Event 

h 

Unit:  D/1-9  CAV 

Go 

No  Go 

Create  Event 

i 

Loc:  ivo  38SM  B383  06040 

Go 

No  Go 

Set  Event  Properties 

j 

Comments:  1  X  WIA  and  1  x  disabled  M1025 

Go 

No  Go 

Set  Event  Properties 
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Role  Name  Map  -  Mosul  Preset 


Task  18  Edit  Graphics 

Comments 

a 

Add  a  Division  Boundary 

Go 

No  Go 

Create  Graphic 

b 

Graphic  Control  Measure:  Boundary  (XX) 

Go 

No  Go 

Create  Graphic 

c 

Label:  ASR 

Go 

No  Go 

Name/Label  Graphic 

d 

Graphic  Control  Measure:  ASR/MSR 

Go 

No  Go 

Name/Label  Graphic 

e 

Loc:  Any  road  on  preset 

Go 

No  Go 

Locate  Product 

Task  19  Create  Effort 

Comments 

a 

Label:  Mosul  Graphics 

Go 

No  Go 

Create.  Name.&  Place  Effort 

b 

Elements:  SAF  Event 

Go 

No  Go 

Create.  Name,&  Place  Effort 

c 

IED  Event 

Go 

No  Go 

Populate  Effort 

d 

Division  Boundary 

Go 

No  Go 

Populate  Effort 

e 

ASR  Graphic 

Go 

No  Go 

Populate  Effort 
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Overlay  PE  Pasteboard  -  Role  Name  BTL  CPT  WS  Pasteboard 


Task  20  Create  Pasteboard 

Comments 

a 

Name:  Role  Name  BTL  CPT  WS 

Go 

No  Go 

Create  and  Name  Pasteboard 

b 

Nested  on  Pasteboard 

Go 

No  Go 

Create  and  Name  Pasteboard 

c 

Map  nested 

Go 

No  Go 

Create,  Name,  &  Nest  Man 

d 

Map  privileges  set  to  any  User  Role  No  Got  in 
CPOF  Class 

Go 

No  Go 

Set  Privileaes 

e 

Annex_C_BDE_FRAGO_01-07 

Go 

No  Go 

PASS:  import  &  display 
product 

f 

MND_I_ANNEXC 

Go 

No  Go 

PASS:  import  &  display 
product 

g 

Auto  Layout 

Go 

No  Go 

Set  Automatic  Layout 

Overlay  PE  Pasteboard  -  Role  Name  BTL  CPT  COP  Pasteboard 


Task  21  Create  Pasteboard 

Comments 

a 

Name:  Role  Name  BTL  CPT  COP 

Go 

No  Go 

Create  and  Name  Pasteboard 

b 

Nested  on  Pasteboard 

Go 

No  Go 

Create  and  Name  Pasteboard 

c 

Map  nested 

Go 

No  Go 

Create,  Name,  &  Nest  Mao 

d 

Pasteboard  privileges  set  to  any  User  Role  No 

Got  in  CPOF  Class 

Go 

No  Go 

Set  Privileaes 

e 

IBCTEthnicity 

Go 

No  Go 

PASS:  import  &  display 
product 

f 

CORPxANNEXIxMSR 

Go 

No  Go 

PASS:  import  &  display 
product 

g 

Auto  Layout 

Go 

No  Go 

Set  Automatic  Layout 

h 

Blue  SA  Effort  on  Map 

Go 

No  Go 

Locate  Product 

i 

Activated  Current  PLI 

Go 

No  Go 

PASS:  import  &  display 
product 
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Overlay  PE  Pasteboard 


Task  22  Nesting  and  Layouts 

Comments 

a 

Master  Schedule 

Go 

No  Go 

Create  Master  Schedule 

b 

Nested 

Go 

No  Go 

Create  Master  Schedule 

c 

Event  Table 

Go 

No  Go 

Set  Event  Table  Prooertles 

d 

Nested 

Go 

No  Go 

Set  Event  Table  Prooertles 

e 

Table 

Go 

No  Go 

Not  scored  in  retention  test 

f 

Nested 

Go 

No  Go 

Set  Event  Table  Properties 

g 

Auto  Layout 

Go 

No  Go 

Set  Automatic  Lavout 

Overlay  PE  Pasteboard 


Task  23  Import  Digital  Photo 

Comments 

a 

Image  Pasteboard  Nested  on  Overlay  PE 
Pasteboard 

Go 

No  Go 

Create  and  Name  Pasteboard 

b 

Display  Image 

Go 

No  Go 

Not  scored  in  retention  test 

c 

Nested 

Go 

No  Go 

Not  scored  in  retention  test 
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APPENDIX  C 

DEMOGRAPHIC  QUESTIONNAIRE 
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FOR  OFFICE  USE  ONLY 

Date _ 

PE  _ 

ID 


Demographic  Information  for  CPOF  Skill  Retention 


General  Instructions:  These  items  deal  with  your  experience  with  digital  systems.  Please  respond  to  each 
item  as  carefully  as  you  can.  If  you  have  questions,  please  feel  free  to  ask  the  researcher. 

1 .  What  is  your  current  grade/rank? 


El  (PV1) 

□ 

E2  (PV2) 

□ 

E3 (PFC) 

□ 

E4  (SPC/CPL) 

□ 

E5  (SGT) 

□ 

E6  (SSG) 

□ 

E7  (SFC) 

□ 

E8  (MSG/1  SG) 

□ 

E9  (SGM/CSM) 

□ 

01/02  (LT) 

□ 

03  (OPT) 

□ 

04  (MAJ) 

□ 

05  (LTC) 

□ 

2.  What  is  your  MOS  (enlisted)  /  branch  (officer)? 

3.  What  is  your  current  duty  position?  _ 

4.  What  is  your  total  time  in  service  (TIS)? _ 
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5.  If  you  have  staff  experience  in  a  Tactical  Operations  Center  (TOC)  or  a  Command  Post  (CP)  at 
any  level,  complete  Item  5.  If  not,  check  here _ and  continue  to  the  next  page. 

5a)  Which  section/element  of  the  staff  were  you  assigned?  CHECK  ALL  THAT  APPLY. 


S/G-2  (Intelligence) 

□ 

S/G-3  (Current  Operations) 

□ 

S/G-3  (Future  Operations) 

□ 

S/G-4/1  (Sustainment  Operations) 

□ 

S/G-5  (Civil  Military  Operations) 

□ 

S/G-6  (C4  Operations) 

□ 

S/G-7  (Information  Operations) 

□ 

5b)  Which  echelon  were  you  assigned?  CHECK  ALL  THAT  APPLY. 


Division 

□ 

Brigade 

□ 

Battalion 

□ 

Company 

□ 

5c)  What  was  your  duty  position  within  the  TOC/CP?  LIST  ANY  THAT  APPLY. 
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6.  Which  of  the  following  courses  have  you  completed?  CHECK  ALL  THAT  APPLY. 


ANCOC 

□ 

U.S.  Army  Sergeants  Major  Academy  (USASMA) 

□ 

Battle  Staff  NCO  Course 

□ 

Maneuver  Captain’s  Career  Course  (MC3) 

□ 

Intermediate  Level  Education  (ILE) 

□ 

School  of  Advanced  Military  Studies  (SAMS) 

□ 

None 

□ 

7.  Which  type  of  experience  do  you  have  using  personal  computers?  CHECK  ALL  THAT  APPLY. 


Used  application  software  (e.g.  Outlook,  PowerPoint,  games) 

Installed  application  software 

Installed  software  patches 

Installed  Hardware  (e.g.  hard  drive,  graphics  card) 

Changed  boot-up  options  or  BIOS  settings 

Authored  web  pages 

Authored  programs 

Never  used  a  personal  computer  before 


8.  Which  personal  computer  applications  have  you  used?  CHECK  ALL  THAT  APPLY. 


Word  Processing  (e.g.,  Microsoft  Word,  WordPerfect) 

□ 

Spreadsheet/Database  (e.g.,  Excel,  Lotus  1-2-3,  Access) 

□ 

Presentations  (e.g.,  PowerPoint,  Keynote,  Adobe  Acrobat) 

□ 

Graphics  (e.g.,  MS  Paintbrush,  Autocad) 

□ 

Layout  (e.g.,  Page  Maker,  Adobe  Dreamweaver) 

□ 

Digital  Photo  Editing  (e.g.,  Kodak  Easyshare,  Adobe  8) 

□ 

None 

□ 
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9.  Overall,  how  would  you  rate  your  proficiency  on  each  of  the  following  digital  systems? 


Basic-  You  can  use  the  system  to  perform  a  limited  set  of  functions  but  there  are  many 
aspects  of  the  system  with  which  you  are  unfamiliar. 

Medium-  You  are  comfortable  with  the  system  and  are  knowledgeable  about  most  of  its 
functions  and  quirks.  You  have  limited  troubleshooting  abilities. 

High-  You  have  advanced  knowledge  of  this  system  and  can  troubleshoot  many 
problems.  You  frequently  are  asked  to  help  others  who  have  difficulty  with  the  system. 


Check  the  appropriate  box  for  each  digital  system. 


System  Never  Basic  Medium  High 

Used 

FBCB2 

□ 

□ 

□ 

□ 

ASAS 

□ 

□ 

□ 

□ 

MCS 

□ 

□ 

□ 

□ 

AFATDS 

□ 

□ 

□ 

□ 

CPOF 

□ 

□ 

□ 

□ 

10.  Prior  to  attending  the  CPOF  Course,  which  formal  individual  system  training  did  you  receive 
for  each  of  the  following  digital  systems?  CHECK  ALL  THAT  APPLY. 


FBCB2  ASAS  MCS  AFATDS  CPOF 

No  Formal  Training 

Online  Course  (self  study) 

Familiarization  training  in  a  classroom 
(typically  1-3  days  of  training) 

Operator  Course  besides  this  CPOF  course 
(typically  5  or  more  days  of  training) 

NET  Training 

NET  Delta  Training  (trained  on 
changes  and  upgrades) 

Digital  Master  Trainer  Course 

Other: 

1 1 .  Prior  to  attending  the  CPOF  Course,  which  types  of  unit  collective  training  did  you  receive  for 
each  of  the  following  digital  systems?  CHECK  ALL  THAT  APPLY. 


FBCB2  ASAS  MCS  AFATDS  CPOF 

No  Unit  Collective  Training 

Motorpool  training 

FTX  at  home  station  or  CTC 

CPX  at  home  station  or  CTC 

Other: 
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12.  Which  of  the  following  Digital/Satellite  Topography  Systems  have  you  used? 

CHECK  ALL  THAT  APPLY. 


Digital  Topography  Support  System  (DTSS) 

□ 

Terrabase 

□ 

Falcon  View 

□ 

Google  Earth 

□ 

None 

□ 

13.  Which  of  the  following  Air  Defense/Airspace  Management  Systems  have  you  used? 

CHECK  ALL  THAT  APPLY. 


Air  and  Missile  Defense  Planning  and  Control  System 
(AMDPCS) 

□ 

Air  Missile  Defense  Warning  System  (AMDWS) 

□ 

Tactical  Airspace  Integration  System  (TAIS) 

□ 

None 

□ 

14.  At  this  time,  which  duty  position/role  would  you  most  likely  use  CPOF  in  an  operational  setting? 


1  probably  would  not  use  CPOF  at  all 

□ 

Primary  Operator  for  a  Staff  Section/Element 

□ 

Staff  Officer  /  NCO  in  BN  or  BDE  TOC 

□ 

BN/BDE/CO  Commander 

□ 
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15.  If  you  used  CPOF  while  deployed  on  a  combat  tour,  please  complete  the  following  for  your  Most 
Recent  Combat  Experience.  If  not,  check  here _ and  continue  to  Item  16. 


Number  of  months 
you  used  CPOF 
during  tour. 

1-6  months 

□ 

7-12  months 

□ 

13-18  months 

□ 

19+  months 

□ 

Combat  Theater 

Iraq  (OIF) 

□ 

Afghanistan  (OEF) 

□ 

Kosovo 

□ 

Other 

□ 

Duty  position(s)  or  role(s)  when 
using  this  system. 

(mark  ail  that  apply) 

Primary  Operator  for  a  Staff 
Section/Element 

□ 

Staff  Officer  /  NCO  in  BN  or 

BDE  TOC 

□ 

BN/BDE  Commander 

□ 

1 6.  If  you  used  FBCB2  while  deployed  on  a  combat  tour,  please  complete  the  following  for  your  Most 
Recent  Combat  Experience.  If  not,  check  here _ and  continue  to  Item  17. 


Duty  position(s)  or  role(s)  when 
using  this  system. 

(mark  all  that  apply) 

Primary  Operator  for  a 

Leader 

□ 

Section  Ldr?  Squad  Ldr 

□ 

Vehicle  Cdr  (other  than 
Ldr/Cdr) 

□ 

PLT  Ldr  /  PSG 

□ 

Co  Hqs  or  support  element 

□ 

Co  Cdr  /  T roop  CDR  /  1 SG 

□ 

Staff  Officer  /  NCO 

□ 

Combat  Theater 

Iraq  (OIF) 

□ 

Afghanistan  (OEF) 

□ 

Kosovo 

□ 

Other 

□ 

Number  of  months 
you  used  FBCB2 
during  tour. 

1-6  months 

□ 

7-12  months 

□ 

13-18  months 

□ 

19+  months 

□ 

17.  If  you  used  ASAS  while  deployed  on  a  combat  tour,  please  complete  the  following  for  your  Most 
Recent  Combat  Experience.  If  not,  check  here _ and  continue  to  Item  18. 


Combat  Theater 

Iraq  (OIF) 

□ 

Afghanistan  (OEF) 

□ 

Kosovo 

□ 

Other 

□ 

Duty  position(s)  or  role(s)  when 
using  this  system. 

(mark  all  that  apply) 

Primary  Operator  for  an 

Intelligence 

Staff/Section/Element 

□ 

Staff  Officer /NCO 

□ 

Number  of  months 
you  used  ASAS 
during  tour. 

1-6  months 

□ 

7-12  months 

□ 

13-18  months 

□ 

19+  months 

□ 
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1 8.  If  you  used  AFATDS  while  deployed  on  a  combat  tour,  please  complete  the  following  for  your 
Most  Recent  Combat  Experience.  If  not,  check  here _ and  continue  to  Item  1 9. 


Combat  Theater 

Iraq  (OIF) 

□ 

Afghanistan  (OEF) 

□ 

Kosovo 

□ 

Other 

□ 

Duty  position(s)  or  role(s)  when 
using  this  system. 

(mark  all  that  apply) 

Primary  Operator  for  a  Fire 
Support  Element  (FSE) 

□ 

Section  Leader 

□ 

Battery  HQ  or  support 
element 

□ 

Battery  CDR/  1SG 

□ 

Staff  Officer  /  NCO 

□ 

Number  of  months  you 
used  AFATDS  during 
tour. 

1-6  months 

□ 

7-12  months 

□ 

13-18  months 

□ 

19+  months 

□ 

1 9.  If  you  used  MCS  while  deployed  on  a  combat  tour,  please  complete  the  following  for  your  Most 
Recent  Combat  Experience.  If  not,  check  here _ . 


Combat  Theater 

Iraq  (OIF) 

□ 

Afghanistan 

(OEF) 

□ 

Kosovo 

□ 

Other 

□ 

Duty  position(s)  or  role(s)  when 
using  this  system. 

(mark  all  that  apply) 

Primary  Operator  for  a  Staff 
Section/Element 

□ 

Co  HQ  or  support  element 

□ 

Co  CDR /Troop  CDR/1SG 

□ 

Staff  Officer /NCO 

□ 

Number  of  months  you 
used  MCS  during  tour. 

1-6  months 

□ 

7-12  months 

□ 

13-18  months 

□ 

19+  months 

□ 

Survey  is  complete.  Please  return  form  to  researcher. 
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APPENDIX  D 

RETENTION  QUESTIONNAIRE 
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FOR  OFFICE  USE  ONLY 

Date 

PE 

RETENTION 

ID 

CPOF  Training  Questionnaire 


General  Instructions:  These  items  deal  with  your  experience  with  digital  systems  since  you’ve 
completed  CPOF  training  at  BCTC.  Please  respond  to  each  item  as  carefully  as  you  can  and  remember 
to  only  include  information  since  your  CPOF  training  BCTC.  If  you  have  questions,  please  feel  free  to 
ask  the  researcher. 

1.  Please  indicate  your  current  rank/grade  below. 


El  (PV1) 

□ 

E2  (PV2) 

□ 

E3 (PFC) 

□ 

E4  (SPC/CPL) 

□ 

E5  (SGT) 

□ 

E6  (SSG) 

□ 

E7  (SFC) 

□ 

E8  (MSG/1  SG) 

□ 

E9  (SGM/CSM) 

□ 

01/02  (LT) 

□ 

03  (CPT) 

□ 

04  (MAJ) 

□ 

05  (LTC) 

□ 

2.  Have  you  changed  your  MOS  (enlisted)  /  branch  (officer)  since  completing  the  CPOF  training  at 
BCTC?  If  so,  please  indicate  your  new  MOS  below. 


3.  Have  you  changed  your  duty  position  since  completing  the  CPOF  training  at  BCTC?  If  so, 
please  indicate  your  new  duty  position  below. 
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4.  If  you  have  any  new  staff  experience  in  a  TAC,  TOC,  or  CP  at  any  level  since  completing  the 

CPOF  training  at  BCTC,  complete  Item  4.  If  not,  check  here _ and  continue  to  the  next 

page. 

4a)  What  section/element  is  your  new  staff  experience?  CHECK  ALL  THAT  APPLY. 


S/G-2  (Intelligence) 

□ 

S/G-3  (Current  Operations) 

S/G-3  (Future  Operations) 

□ 

S/G-4/1  (Sustainment  Operations) 

□ 

S/G-5  (Civil  Military  Operations) 

□ 

S/G-6  (C4  Operations) 

□ 

S/G-7  (Information  Operations) 

□ 

4b)  What  echelon  is  your  new  staff  experience?  CHECK  ALL  THAT  APPLY. 


Corps 

□ 

Division 

Brigade 

Battalion 

Company 

□ 

4c)  What  is  your  new  duty  position  within  the  TAC/TOC/CP? 

LIST  ANY  THAT  APPLY. 
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5. 


Since  attending  the  CPOF  Course  at  BCTC,  did  you  receive  any  formal  individual-system 
training  on  the  following  digital  systems?  If  so,  indicate  the  system  and  type  of  training  below. 


CHECK  ALL  THAT  APPLY. 


FBCB2  ASAS  MCS  AFATDS 

No  Formal  Training 

Online  Course  (self  study) 

Familiarization  training  in  a  classroom 
(typically  1-3  days  of  training) 

Operator  Course  besides  this  CPOF  course 
(typically  5  or  more  days  of  training) 

NET  Training 

NET  Delta  Training  (trained  on 
changes  and  upgrades) 

Digital  Master  Trainer  Course 

Other: 

6. 


Since  attending  the  CPOF  Course  at  BCTC,  did  you  receive  any  unit  collective  training  on  the 
following  digital  systems?  If  so,  indicate  the  system  and  type  of  training  below. 


CHECK  ALL  THAT  APPLY. 


FBCB2  ASAS  MCS  AFATDS 


No  Unit  Collective  Training 

Motorpool  training 

FTX  at  home  station  or  CTC 

CPX  at  home  station  or  CTC 

Other: 

7.  Since  completing  your  CPOF  training  at  BCTC,  did  you  participate  in  a  Field  Training  Exercise 
(FTX)  in  which  CPOF  was  used?  If  not,  check  here _ and  continue  to  Item  8. 

7a)  Indicate  how  you  used  CPOF  during  the  FTX.  CHECK  ALL  THAT  APPLY. 


Saw  CPOF  being  used 

□ 

Attended  or  participated  in  BUB/CUB 

□ 

Constructed  or  helped  construct  a  BUB/CUB 

□ 

Primary  Operator  in  a  TOC/CP 

□ 

Battle  CPU  Battle  NCO 

□ 

S2 

□ 

Future  Operations  cell 

□ 

Other 

□ 
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8.  Since  completing  your  CPOF  training  at  BCTC,  did  you  participate  in  a  Command  Post  Exercise 
(CPX)  in  which  CPOF  was  used?  If  not,  check  here _ and  continue  to  Item  9. 

8a)  Indicate  how  you  used  CPOF  during  the  CPX.  CHECK  ALL  THAT  APPLY. 


Saw  CPOF  being  used 

□ 

Attended  or  participated  in  BUB/CUB 

□ 

Constructed  or  helped  construct  a  BUB/CUB 

□ 

Primary  Operator  in  a  TOC/CP 

□ 

Battle  CPT/  Battle  NCO 

□ 

S2 

□ 

Future  Operations  Cell 

□ 

Other 

□ 

9.  Since  completing  your  CPOF  training  at  BCTC,  how  often  did  you  use  CPOF? 


Never 

□ 

Once  or  twice 

□ 

A  few  times 

□ 

At  least  once  a  week 

□ 

Daily 

□ 

10.  Overall,  how  would  you  rate  your  proficiency  on  CPOF  at  this  point? 


Poor 

Low 

Good 

High 

□ 

□ 

□ 

□ 

Survey  is  complete.  Please  return  form  to  researcher. 
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